This application relates the field of computer networking and systems and methods for processing network traffic. Network administrators often employ numerous different types of network devices to intercept network traffic between two or more entities on a communications network. These intercepting network devices may be used to monitor, optimize, filter, block, convert, or otherwise process the network traffic.
For example, many organizations operate at numerous different locations. Typically, these organizations use a separate local area network (LAN) at each location. The LANs for different locations are then connected by a wide area network (WAN) that enables users to access data located anywhere in the organization. Typically, the WAN connection is the weakest link in the organization's network infrastructure, having the slowest speed, the highest cost, and the lowest reliability.
To overcome many of these problems, organizations can use network acceleration appliances that improve WAN performance and reliability and reduce WAN utilization (and hence cost). Among the capabilities of network acceleration appliances is the ability to intercept and optimize network traffic at the TCP level. This allows appliances to address the impact of latency on TCP windowing mechanisms and allows for greater utilization of Wide Area Networks (WANs).
However, intercepting network devices such as network acceleration appliances often have limits on the amount of network traffic that can be intercepted and processed. For example, some network acceleration appliances have a limit to the number of TCP flows that can be handled, or other capacity constraints. Typically, when the number of TCP flows or other processing demands is expected to exceed the capacity of a single network device, network engineers split the network traffic into parallel data flows to a set of two or more intercepting network devices, such that each intercepting network device handles a subset of network traffic, and in combination, the set of intercepting network devices handles the full network traffic.
Unfortunately, splitting network traffic into two or more parallel data flows causes additional network complexity that often needs to be compensated for so that the network continues to operate correctly. In addition, such network traffic splitting requires accurate predictions of the network traffic to be effective and ensure that the network traffic load is evenly distributed. There is no straightforward way to simply add extra intercepting network devices to increase network traffic capacity and provide a safety margin of excess network capacity.
It is therefore desirable for a system and method to enable networks to easily increase the network traffic capacity of intercepting network devices, such as network acceleration appliances, without introducing undesirable network complexities. It is further desirable for the system and method to be easy to deploy and configured compared with prior network scaling techniques. Finally, although the motivation presented here is in terms of network acceleration appliances and limits on TCP flows, it is desirable for the system and method to be applicable to any type of intercepting network device and general network traffic, including network traffic using UDP or other datagram-oriented protocols.